Vehicle surroundings image conversion devices have been used in driver support systems for assisting and aiding the vision of a driver in situations when a driver is backing a vehicle (such as parking in a garage or inching close to another vehicle) or driving forward and entering an intersection where visibility is poor or a T-shaped intersection.
The places imaged by the camera connected to such a vehicle surroundings image conversion device are, for example, behind the vehicle for the purpose of displaying a rear view, and the left and right directions from the front of the vehicle for the purpose of displaying blind corner views. There is a need to simultaneously present views in multiple directions to the driver. Hence, a driving support system utilizes multiple cameras imaging in various differing directions picking up images of the surroundings of a vehicle, and by virtue of a vehicle surroundings image conversion device arranges and displays multiple images on a single screen of a monitor and presents them to a driver.
In a vehicle surroundings image conversion device like this a technology for synchronizing the images picked up by the multiple cameras has been described in Official Gazette of Japanese Unexamined Patent Application Hei 11-317908. This approach connects multiple cameras and a controller, and by virtue of this controller creates a synchronizing signal and sends it to each camera, and each camera in turn sends an image signal to the controller in accordance with this synchronizing signal. The controller then performs image conversion processing of the image signals that the multiple cameras have picked up and then displays the processed images on a monitor.
The inventors recognized that this prior art approach is complex, because the configuration for the controller needs to create a synchronizing signal and provide a signal line for supplying the synchronizing signal to each camera. In contrast to this prior art approach, there is a technology with a simple configuration which does not utilize a synchronizing signal to synchronize the image signals picked up by each camera, but rather, receives an image signal from each camera and synchronizes each of the image signals by accumulating these respective image signals temporarily in a buffer. For example, the following will explain the resolution of an image data picked up by the respective cameras and the resolution of the monitor presenting it to the driver as VGA size (Video Graphics Array, 640×480 picture elements). Likewise, the following will explain the updating of data presented to the driver when processing in 1 frame units.
Each camera sends an image signal according to the timing of its respective internal clock, and the vehicle surroundings image conversion device has 3 input buffers which have 1 frame of memory volume (address space of 640×480 picture elements) on each buffer's input side and output side for each camera system connected. That is to say, the vehicle surroundings image conversion device has an input buffer with 3 frames of memory volume by way of an input phase, a standby phase and an output phase (each having 1 frame of memory volume). By virtue of this, when an image signal of 1 frame from a certain camera has completed input to the input phase, the device stores this image signal of 1 frame in the standby phase, and when image signals of 1 frame are stored in the standby phase of input buffers associated with all cameras, the device moves the image signals of all cameras from the standby phase to the output phase. Then the vehicle surroundings image conversion device simultaneously reads the multiple image signals stored in the output phase and performs image conversion for displaying them on a monitor, for example. By doing this, even though each of the cameras is not synchronized, the device matches the reading phase of image signals, absorbs the asynchronous input of multiple cameras and performs synchronization.
At this time the CPU (Central Processing Unit) in the vehicle surroundings image conversion device endeavors to synchronize the image signals by rotating the 3 phases of the input buffer. Then the CPU reads the image signal stored in the side designated for address conversion processing, in other words the output side of the input buffer, in frame units according to the input buffer reading address stored in pattern memory, performs address conversion processing, creates an image signal for displaying on a monitor and stores this in an output buffer. Incidentally, the address space provided in pattern memory is equivalent to the resolution of the monitor which presents the image after address conversion to the driver and in this case would be 640×480 picture elements.
However, with the technology described above, because the image signal input from each camera is processed in single frame units, a delay of a maximum of approximately 2 frames occurs in order to obtain synchronization between respective image signals in the input buffer. Additionally, if address conversion processing is performed and the timing for displaying an image is synchronized with the timing stored of images from all cameras in the output side of the input buffer, a delay of a single frame occurs. Thus a delay of a total of 3 frames will occur from the time the image signal is stored in the input side of the input buffer to the time that it is stored in the output buffer.
How a maximum of a delay of 3 frames occurs in this way will now be explained with reference to FIG. 7. For example, when 2 NTSC (National Television System Committee) cameras (camera A, camera B) are connected to a vehicle surroundings image conversion device, posit that from time t101 image data from camera A (input data 1a) begins to be received in the input side of an input buffer and later than this time t101 image data from camera B (input data 1b) begins to be received in an input side of an input buffer (FIGS. 7(b), (c)). If one posits that these cameras A and B are the known imaging type NTSC type (frame rate 29.97 fps) and the time that the input data of 1 frame of camera A and camera B have finished being stored in the output sides of the respective input buffers is t102, the delay time required for input will be approximately 2 frames. Then, the respective input data from the output side of the input buffer is read at this time t102, conversion processing begins (FIG. 7(d)) and the time that converted data is actually finished being stored in an output buffer as output data will be t103 (FIG. 7(e)). Thus, the total delay time from the time that input data of camera A begins to be input, t101, to the time that output data is output to a monitor etc. and displayed, t103, will be approximately 100 msec.
As a result of this total delay time, a vehicle equipped with a vehicle surroundings image conversion device ascertaining an object moving at a relative speed of 36 km will cause the obstacle to be displayed approximately 1 m from the actual position of the obstacle. While some delay can be permissible when using a vehicle surroundings image conversion device in lower speed ranges, such as parking and inching closer to another vehicle, it is unacceptable when using it when entering an intersection with poor visibility or a T-shaped intersection. In addition, in a situation with moderate to high speeds such as passing between two cars or converging, the discrepancy between the position of the obstacle which is displayed and the actual position of the obstacle will increase even more.
Moreover, when the discrepancy between the position of the obstacle which is displayed and the actual position of the obstacle increases, problems occur such as when one's vehicle has begun to move and although the actual vehicle is moving the picture on the monitor is not moving, or when one's vehicle has stopped and the actual vehicle is not moving but the picture on the screen is moving. This inconsistency between the movement of the vehicle and the movement of the picture on the monitor makes the driver feel uncomfortable. In this way, the delay of the presentation of the picture to the driver is a serious problem when multiple cameras are utilized.
Likewise, as the number of cameras attached to the previously described vehicle surroundings image conversion device proliferates, the volume of the input buffer memory increases 3 frames for each one.